Skip to content

nvidia-peermem: Prefer ofa_kernel-dkms path for DOCA-OFED headers#1018

Open
blochl wants to merge 2 commits intoNVIDIA:mainfrom
blochl:lb-dev
Open

nvidia-peermem: Prefer ofa_kernel-dkms path for DOCA-OFED headers#1018
blochl wants to merge 2 commits intoNVIDIA:mainfrom
blochl:lb-dev

Conversation

@blochl
Copy link

@blochl blochl commented Feb 3, 2026

New DKMS-based DOCA-OFED installations store headers in /usr/src/ofa_kernel-dkms/ and create symlinks from /usr/src/ofa_kernel/ for backward compatibility.

Using the ...-dkms path directly is more robust as it contains the actual files, avoiding potential issues with symlinks that may be missing, broken, or not yet created during package installation.

@CLAassistant
Copy link

CLAassistant commented Feb 3, 2026

CLA assistant check
All committers have signed the CLA.

@Binary-Eater
Copy link
Collaborator

@blochl I believe the original version for specifying the DOCA-OFED kernel path might be better. The main reason I feel this way is I am worried about the scenario where after installing a DKMS kernel, the user decides to install the non-DKMS kernel without cleaning up the previous DKMS kernel install. I believe that can leave /usr/src/ofa_kernel-dkms/ residue while /usr/src/ofa_kernel/ points to the correct source.

blochl added 2 commits March 11, 2026 16:03
The kernel build system normalizes x86_64 to x86 via
scripts/subarch.include, but MOFED directories use the `uname -m`
name x86_64. Add the missing mapping, consistent with the existing
arm64 -> aarch64 and powerpc -> ppc64le entries.

Without this, OFA_ARCH is set to x86 on x86_64 systems.

Signed-off-by: Leonid Bloch <lb.workbox@gmail.com>
Since DOCA 3.2.0, DKMS is the default OFED drivers' installation
method. These store the headers in /usr/src/ofa_kernel-dkms/ and
create symlinks from /usr/src/ofa_kernel/ for backward
compatibility.

Using the -dkms path directly is more robust as it contains the
actual files, avoiding potential issues with symlinks that may be
missing, broken, or not yet created during package installation.

Add ofa_kernel-dkms as the highest-priority candidate, drop the
/var/lib/dkms/mlnx-ofed-kernel/ fallback whose directory structure
never matched the expected layout, and replace the shell for-loop
with $(firstword $(wildcard ...)) to match the pattern used by
OFED's own DKMS submodules.

Signed-off-by: Leonid Bloch <lb.workbox@gmail.com>
@blochl
Copy link
Author

blochl commented Mar 11, 2026

@Binary-Eater, Thanks for the review!
The DOCA DKMS packages remove the -dkms directory on removal (in post-remove). But in any case, I have reworked the commit to be more robust and to use the same logic as the DOCA OFED itself uses.
Plus, I found an issue with x86 that would have manifested with DKMS modules, and fixed it in a separate commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants